Digital image analysis, processing, compression and decompression means are key features in adjusting or enhancing the quality of images that are captured, received and sent using mobile terminals. In mobile imaging, the ‘image chain’ is mainly comprised of operations performed on the pixels within an image or a sequence of images, beginning for example with the capture of the image and extending up to the decompression of the image in order to display it, so that people can see the image correctly on, for example, a mobile terminal screen. The mobile terminal is, for example, a cell phone or a mobile camera phone, often called a ‘phonecam’. This means that a phonecam can be used to capture an image or a sequence of images, after which this image or sequence of images can then be made to undergo various pixel processing processes. Processing a color image captured using a phonecam that is equipped with a CCD sensor consists, for example, in a CFA (Color Filter Array) interpolation using an algorithm in order to process each pixel in the image captured, taking into account primary color components, i.e. red, green and blue. The image can then be compressed to make it lighter in computer terms, the image weight being expressed in megabytes or gigabytes. The compression routine run may, for example, be a JPEG compression, which is a standardized compression procedure. Then, the image is decompressed when the compressed file is opened in order to view it correctly, for example on the screen of the phonecam, and possibly to listen to any sound data associated with the image data. In mobile imaging, all the abovementioned operations are routine image chain operations.
In addition to these operations performed in the image chain operation performed on pixel data, mobile terminals can also be equipped with data sensors designed to acquire and process a range of different contextual data associated with the images captures. The mobile terminal may, for example, be a phonecam equipped with a GPS module to gauge the geographical location of the terminal; this geographical location is a contextual data characterizing, for example, the place where the image was captured. This contextual data is frequently used as one of the image metadata.
However, these contextual data are only associated with the image as simple metadata, meaning that it does not play a role in actively improving the process of capturing a sequence of images, or more specifically, the process of capturing an image of the displayed sequence, for example, is being viewed at the same time on the screen of the mobile terminal.
Furthermore, mobile or portable terminals such as camera-phones or phonecams are generally relatively compact, and so consequently they have only limited energy and memory capacities. The energy capacity remains limited because the mobile terminal general has to carry its own energy, which means it is equipped with a low-capacity battery.
The process of efficiently working on images captured by digital camera devices in order to extract relevant data, for example the place where the shot was taken from, the people featuring in the shot or the action taking place in the scene of the photo, can be made easier by having access to additional data such as metadata or contextual data. This data makes it possible to calculate or directly extract key information on the semantic content of an image.
The best way to capture a context would be to continuously monitor that context both before and after the image capture: the quantity of contextual data associated with the image capture would then ensure that no important event would be missed later on in the ‘image chain’. This is the underlying principle behind the electronic surveillance systems used in the security industry, where multiple sensors (visual, infrared, audio, etc.) are integrated into an environment to track and analyze a same scene. However, this kind of approach would not be viable if it had to be integrated into a mobile terminal, due to the terminal's energy consumption and storage and processing capacity of said terminal. More importantly, a mobile image capture terminal such as, for example, a phonecam or a digital camera, is only one component in the ‘image chain’; consequently, the resulting increase in the data flow to be processed in the communication channel at every link in the chain (visual display, storage, print-out) would make it impractical to employ this kind of continuous contextual data acquisition process, for performance reasons and in terms of easiness to use of the mobile capture terminal.
Therefore, there is a need to reduce the quantity of contextual data to be processed while at the same time maintaining a high level of quality to allow the efficient exploitation of said data.
There is therefore an overriding need, given the limitations inherent to mobile terminals capable of producing shots, to optimize images capture and rendering using a capture device such as a phonecam or a cellphone equipped, for example, with an image sensor and contextual data sensors. One solution can consist in exploiting information coming from the contextual data associated with the images capture. Starting out with these contextual data, that leads to interpret the contextual data in order to capture the images in good conditions, by triggering, for example, the capture of one image with a specifically tailored pixel resolution.
The contextual data are intended to be used to optimize the management of an image or of a sequence of images capture, by taking into account the context, which may be progressive or fast-changing in an mobile imaging environment, while at the same time fitting the relatively modest memory and energy capacities of mobile image capture terminals. The contextual data can, therefore, be acquired and interpreted early on, for example during the visualization phase carried out using the phonecam's display, i.e. before an image is captured.
The optimization of the images capture, from the provision of contextual data has to resolve different problems, in regard to the power and calculation and memory capacity constraints inherent to mobile platforms or terminals capable of images capture. There are many different problems that have to be resolved: the management of contextual data associated with the images capture, i.e. the quality and sampling process of the contextual and images data captured over time; the image capture means and the processing means used on contextual data associated with the images captured; the saving and the storage of the images data and the associated contextual data; the means of processing contextual data, given the capacity constraints of mobile terminals capable of simultaneously capturing images and contextual data.